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The MAILING DATE of this communication appears on the cover sheet with the correspondence address « 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1704(b). 

Status 

1 )[>3 Responsive to communication(s) filed on 01 July 2003 . 
2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) [3 Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1^28 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)DAII b)D Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
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DETAILED ACTION 



1. 



This action is responsive to communications: Request for CPA and Preliminary 



Amendment B, filed 7/1/2003. 

2. Claims 1-28 are pending in the case. Claims 1, 7, 10, 13, 20, 25 and 28 are independent 
claims. 

3. Claims 1-27 remain rejected over the prior art. New claim28 stands rejected. 



4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

5. Claims 10-12 and 25-28 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non- functional descriptive data stored on a medium. Although the independent 
claim recites "data structure" in the preamble (or as part of a storage medium), the body of the 
claim merely recites a collection of fields. Use of the broad phrase "correlated through" is 
insufficient to establish the necessary functional interrelationship between elements to constitute 
a true data structure. As such, the claims recite nori- functional descriptive data stored on a 
medium. 



6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



Claim Rejections - 35 USC §101 



Claim Rejections - 35 USC §103 
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7. Claims 1-12, 20, 22, 23, and 25-28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Synchronized Multimedia Integration Language (SMIL) 1.0 
Specification," W3C Recommendation 15-June-1998, herein referred to as W3C, in view of 
deVries et al. (USPN 6332144— (priority date 3/11/1998). 

Regarding independent claim 1, W3C discloses a system (p.26, ex, 4) for associating 
annotations, contained in the <text . . . /> tag, to the plurality of media streams, representing 
different versions of multimedia content, that would be contained in ellipses found at lines 6, 9, 
and 12 of the example, as explained in the header of the example. While there is no explicit 
mention of annotations and multimedia servers, W3C does however disclose that both the 
annotations and media are referenced by a URI (p. 19, a src"), which were well-known to refer to 
servers. deVries et al. (deVries) discloses that media and annotations can be stored on servers 
(FIG IB, 22 and 26), the annotation server inherently having a database of annotations. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine deVries and W3C in order to effectively store the annotations (col. 14, 11. 
5-8), media (col. 4, 11. 34-08), and relations between the two (FIG. 7). 

Regarding dependent claim 2, deVries discloses the streams can be audio or video (col. 1, 11. 



Regarding dependent claim 3, W3C discloses different time compressed versions (p. 26, ex. 2). 
Regarding dependent claim 4, W3C discloses versions having different resolutions (p. 26, ex. 



Regarding dependent claim 5, W3C discloses an identifier for each of the plurality of media 
streams (p. 19, "src"). 



59-61). 



4). 
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Regarding dependent claim 6, W3C discloses the identifier is a URI (p. 19, "src") equivalent to 
a URL. 

Regarding independent claim 7, W3C discloses an SMIL document that correlates an 
annotation with different versions of the same multimedia content as described in the rejection of 
claim 1 . It is the position of the Office that a data structure is equivalent to an XML document 
such as a SMIL. W3C does not disclose storing the annotation. deVries discloses a storage 
device to store the annotation (FIG. IB, 26). It would have been obvious to combine deVries 
and W3C to store the annotation so that it is correlated through the data structure by storing the 
annotation in the storage device at the location referenced by the URI in the <text/> tag. 
Regarding dependent claim 8, W3C discloses aversion list identifying multiple different 
versions (p.26, ex. 4). 

Regarding independent claims 10, 25 and 28, W3C teaches a data field representing an 
annotation corresponding to multimedia content, and a data field correlated so that the field 
identifies a plurality of different versions of the multimedia content to which the annotation 
corresponds (p.26, ex. 4). 

Regarding dependent claims 11 and 26, W3C teaches that identifiers are intended to be 
included at the ellipses in example 4, on page 26. 

Regarding dependent claim 9, 12, and 27, W3C teaches that URI's, equivalent to URL's are 
intended to be included at the ellipses in example 4, on page 26. 

Regarding independent claim 20, W3C discloses receiving an indication of the version of 
media content (p. 18, 11 1-3). W3C also discloses that annotations are corresponding to a 
plurality of different versions of the media content (p. 26, ex. 4). W3C is silent on providing the 
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annotations to he the user. DeVries discloses providing presenting media and annotations to a 
user (col. 19, 11. 29-42). It would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine W3C and deVries as the markup language of W3C was intended to 
describe how media should interact with each other when it is displayed. 
Regarding dependent claim 22, while neither deVries nor W3C explicitly teach comparing 
identifiers, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to compare the identifiers to find out what media the version was a part of 
Regarding dependent claim 23, the computer programs for executing the methods of claims 20 
are rejected under the same rationale. 

8. Claims 13-15, 18-19, and remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over W3C in view of Logan et al. (USPN 5735216-^atented 3/24/1998), 
Regarding independent claim 13, W3C teaches associating an annotation with a set of media 
streams, however, is silent as to the creation of the annotations. Logan teaches receiving a user 
request to create a new annotation (col. 32, 11. 53-56). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to include a method of creating a new 
annotation, as it would be allow easier addition of annotations. 

Regarding dependent claim 14, W3C does not explicitly disclose associating a new record with 
a previously generated record. W3C does disclose annotations in <par> and <seq> tags (sees. 
4.2. 1-. 4. 1.2). Upon the creation of new annotation as recited in claim 13, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to associate the annotation 
within the tags to allow the annotation to be synched with the media stream. 
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Regarding dependent claim 15, it would have been inherent to identify the streams before 
associating anything with them. 

Regarding dependent claim 18, W3C discloses a record of the set of media streams (p. 26, 
ex.4). It would have been obvious to one of ordinary skill in the art at the time of the invention 
to generate and store this record so that it can be read later. 

Regarding dependent claim 19, the computer programs for executing the methods of claims 13 
are rejected under the same rationale. 

9. Claims 16-17, and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
W3C and Logan as applied to claim 13 above, and further in view of deVries. 
Regarding dependent claim 16, while there is no explicit mention of multimedia servers in 
W3C and Logan, W3C does however disclose that both the annotations and media are referenced 
by a URI (p. 19, "src"), which were well-known to refer to servers. deVries et al. (deVries) 
discloses that media and annotations can be stored on servers (FIG IB, 22 and 26), the 
annotation server inherently having a database of annotations. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine deVries and 
W3C in order to effectively store the annotations (col. 14, 11. 5-8), media (col. 4, 11. 34-08), and 
relations between the two (FIG. 7). 

As the annotations would have been stored on the server, it would have been obvious to 
communicate with server and identify the streams before associating anything with them. 
Regarding dependent claim 17, W3C discloses a set of identifiers, each identifier uniquely 
identifying one of the sets of streams (p. 26, ex. 4). 
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Regarding dependent claim 24, W3C and Logan are silent on providing annotations to a user. 
DeVries discloses providing presenting media and annotations to a user (col. 19, 11. 29-42). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to 
combine W3C, Logan and deVries as the markup language of W3C was intended to describe 
how media should interact with each other when it is displayed. 

10. Claim 21 remains rejected under 35 U.S.C. 103(a) as being unpatentable over W3C 
and deVries as applied to claim 20 above, and further in view of Media Weaver, by Sha Xin 
Wei (6/2/1997). 

Regarding dependent claim 21, deVries discloses that annotations are associated with a time 
(col. 16, 11. 42-54). Both W3C and Logan are silent as to converting media. Xei discloses 
converting media (p. 12 bullet 3). It would have been obvious to one of ordinary skill in the art at 
the time of the invention modify Xei into W3C and Logan so that the annotations would still be 
presented in the same context as in the original media. 



1 1 . Applicant's arguments filed 7/1/03 have been fully considered but they are not 
persuasive. 

Regarding Applicant's remarks under 35 USC 101: 

Applicant has repeated the same argument, so the Office has repeated the same response: While 
Applicant is claiming a "data structure" in the preamble; the body of the claim merely recites a 
collection of fields. Use of the broad phrase "correlated through" is insufficient to establish the 
necessary functional interrelationship between elements to constitute a true data structure. As 
such, the claims recite hon- functional descriptive data stored on a medium. 



Response to Arguments 
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Regarding Applicant's remarks on p.13-14 regarding claims 1, 7, 10, and 25: 

Applicant argues that the text tag does not qualify as annotations. The Office disagrees. The 
text is contained with a <par> tag (§ 4.2. 1), with the <switch> block. Within the switch block, as 
noticed in the example's text, are "alternative parts designed for screens with different 
resolutions, and bit-depths." The ellipses at 6, 9, and 12, are clearly meant to be the alternative 
parts. These are the different versions of multimedia content. They would be represented by one 
of the tags disclosed in §4.2.3. Both those tags, and <text> when contained within a <par> can 
be associated to particular time frames (p. 12). While W3C indeed does not make mention of the 
word "annotation," it is still the position of the Office, that text associated with media is an 
annotation. The title of section that Applicant points out is referring to the elements within the 
switch. The ellipses within the text tag do not only indicate that the source is arbitrary, but also 
that any attribute could be contained within, such as the ones listed in §4.2.3. 
Applicant also alleges the Office has admitted, on page 7 of the previous response that the W3C 
is silent with respect to annotations. That is not an accurate assessment of what is said in this 
and the previous action. The Office merely acknowledged that the word "annotation" was not 
used. Just because the same claim language is not used, does not mean that W3C is absent of 
annotations. The Office has explained in the rejection and with the above argument why W3C 
does in fact teach annotations. 

In regard to the remarks on the deVries patent, Applicant merely recites the claim and alleges the 
source does not teach the claim. This does not constitute a valid argument because they amount 
to a general allegation that the claims define a patentable invention without specifically pointing 
out how the language of the claims patentably distinguishes them from the references. 
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Additionally, though deVries has not been relied on for teaching that an annotation is associated 
with multiple versions of multimedia content, Applicant alleges that it does not contain such a 
teaching. Despite the one section of deVries the Applicant quotes, the entire specification, 
exemplified by Fig. 7, teaches annotation is associated with multiple versions of multimedia 
content. 

Regarding dependent claim Applicant's remarks regarding claim 13, 

Applicant merely recites the claim and alleges the source does not teach the claim because the 
source does not contain the word "annotation." Applicant argues that the text tag does not 
qualify as annotations. The Office disagrees. The text is contained with a <par> tag (§ 4.2. 1), 
with the <switch> block. Within the switch block, as noticed in the example's text, are 
"alternative parts designed for screens with different resolutions, and bit-depths " The ellipses at 
6, 9, and 12, are clearly meant to be the alternative parts. These are the different versions of 
multimedia content. They would be represented by one of the tags disclosed in §4.2.3. Both 
those tags, and <text> when contained within a <par> can be associated to particular time frames 
(p. 12). While W3C indeed does not make mention of the word "annotation," it is still the 
position of the Office, that text associated with media is an annotation. The title of section that 
Applicant points out is referring to the elements within the switch. The ellipses within the text 
tag do not only indicate that the source is arbitrary, but also that any attribute could be contained 
within, such as the ones listed in §4.2.3. 

Regarding dependent claim Applicant's remarks regarding claim 20, 

Applicant merely recites the claim and alleges the source does not teach the claim. This does not 
constitute a valid argument because they amount to a general allegation that the claims define a 
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patentable invention without specifically pointing out how the language of the claims patentably 
distinguishes them from the references. However, W3C does teach annotations. 
Applicant argues that the text tag does not qualify as annotations. The Office disagrees. The 
text is contained with a <par> tag (§ 4.2. 1), with the <switch> block. Within the switch block, as 
noticed in the example's text, are "alternative parts designed for screens with different 
resolutions, and bit-depths ." The ellipses at 6, 9, and 12, are clearly meant to be the alternative 
parts. These are the different versions of multimedia content. They would be represented by one 
of the tags disclosed in §4.2.3. Both those tags, and <text> when contained within a <par> can 
be associated to particular time frames (p. 12). While W3C indeed does not make mention of the 
word "annotation," it is still the position of the Office, that text associated with media is an 
annotation. The title of section that Applicant points out is referring to the elements within the 
switch. The ellipses within the text tag do not only indicate that the source is arbitrary, but also 
that any attribute could be contained within, such as the ones listed in §4.2.3, As the missing 
feature has been provided, the Office's rejection does establish a prima facie case of 
obviousness. 

Regarding Applicant's remarks on pp. 17-18, regarding claims 2-6, 8, 9, 11, 12, 14-19, 21- 
24, and 26-27: 

Applicant alleges these claims are allowable in view of their base claims being allowable. As the 
rejections have been maintained and no substantive arguments have been provided as to the 
dependent claims patentability, the claims remain rejected. 
Regarding Applicant's remarks on 18-19, regarding Examiner's Response: 
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The Office sees no deficiencies its prior response to Applicant's previous arguments. The Office 
responded it saw as the two points of the Applicants argument: Statutory subject matter, and the 
interpretation of W3C. The Applicant makes no allegations of what arguments in particular were 
deficient. Applicant alleges repeatedly that the Office has not addressed all arguments, and 
various deficiencies within the arguments without providing any indication of what in particular 
arguments are deficient. As such the Office can not respond to such allegations. The Office has 
however, now added individual responses to the specific claims that were previously addressed 
by the response to arguments regarding claim 1 . 

Additionally the cited portion of the MPEP, specifically the portion labeled Examiner's Note," in 
addition to being located nowhere near the previous cited portion, has been taken completely out 
of context. The Note relates to a form paragraph used by the Office, which has not been used in 
this or the previous rejection. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Adam M Queler whose telephone number is (703) 308-5213, 
The examiner can normally be reached on Monday- Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather R Heradon can be reached on (703) 308-5186. The fax phone number for 
the organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 306-563 1 . 
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ISTEPHEN S. HONG 
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